Skip to content

CI: stop using spot-instances, they are flaky#593

Merged
aramprice merged 1 commit into
ubuntu-jammyfrom
stop-using-spot-instances
May 7, 2026
Merged

CI: stop using spot-instances, they are flaky#593
aramprice merged 1 commit into
ubuntu-jammyfrom
stop-using-spot-instances

Conversation

@aramprice

Copy link
Copy Markdown
Member

These are not stable enough for testing purposes.

NOTE: this repository uses a "Merge Forward" strategy

Changes should be made in the earliest applicable branch, and
merged forward through subsequent branches.

  1. Create a PR into the oldest branch (ubuntu-<short_name>)
  2. After this PR has been merged create a merge-to-<next_short_name> branch
  3. Merge ubuntu-<short_name> into merge-to-<next_short_name>
  4. Create a PR to merge merge-to-<next_short_name> into ubuntu-<next_short_name>
  5. Repeat as needed for subsequent branches

@aramprice aramprice requested review from beyhan, Copilot and ramonskie May 7, 2026 15:56
@linux-foundation-easycla

linux-foundation-easycla Bot commented May 7, 2026

Copy link
Copy Markdown

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: aramprice / name: aram price (0bf1a42)

@coderabbitai

coderabbitai Bot commented May 7, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Rate limit exceeded

@aramprice has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 10 minutes and 31 seconds before requesting another review.

To continue reviewing without waiting, purchase usage credits in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: ccc1a155-2ef9-4c83-ba25-9cf7136ca7fc

📥 Commits

Reviewing files that changed from the base of the PR and between 2c6bc72 and 0bf1a42.

📒 Files selected for processing (1)
  • ci/pipelines/builder.yml

Walkthrough

This pull request disables GCP preemptible instances in two deployment tasks within the pipeline configuration. The test-stemcells-ipv4 and bats jobs have their GCP_PREEMPTIBLE parameter changed from true to false. This affects how these jobs provision GCP resources during their execution.

Suggested reviewers

  • mkocher
  • ystros
  • neddp
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main change: disabling spot/preemptible instances in CI due to instability, which aligns with the changeset.
Description check ✅ Passed The description includes a brief explanation of the change and includes the repository's required merge-forward strategy documentation in full.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch stop-using-spot-instances

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
ci/pipelines/builder.yml (1)

577-594: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

VARS_PREEMPTIBLE: true still enables spot instances for BATs test VMs

The prepare-bats task at line 586 still passes VARS_PREEMPTIBLE: true into the BATs configuration. This controls whether the VMs deployed by BOSH during the BAT test run are preemptible — meaning the actual test VMs remain spot instances and are still subject to the same preemption-driven flakiness the PR is addressing. Only the director was switched to on-demand.

🐛 Proposed fix
-          VARS_PREEMPTIBLE: true
+          VARS_PREEMPTIBLE: false
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/pipelines/builder.yml` around lines 577 - 594, The prepare-bats task is
still passing VARS_PREEMPTIBLE: true which makes the BATs-deployed test VMs
preemptible; change the VARS_PREEMPTIBLE value to false in the prepare-bats task
so BATs creates on‑demand (non‑preemptible) VMs during the test run (update the
VARS_PREEMPTIBLE entry in the same prepare-bats block to false to match the
director change).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@ci/pipelines/builder.yml`:
- Around line 577-594: The prepare-bats task is still passing VARS_PREEMPTIBLE:
true which makes the BATs-deployed test VMs preemptible; change the
VARS_PREEMPTIBLE value to false in the prepare-bats task so BATs creates
on‑demand (non‑preemptible) VMs during the test run (update the VARS_PREEMPTIBLE
entry in the same prepare-bats block to false to match the director change).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 45f6892d-d24a-4e67-8b79-51b751b45f27

📥 Commits

Reviewing files that changed from the base of the PR and between 2d3933d and 2c6bc72.

📒 Files selected for processing (1)
  • ci/pipelines/builder.yml

coderabbitai[bot]
coderabbitai Bot previously approved these changes May 7, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to improve CI stability by disabling GCP preemptible (spot) instances in the builder pipeline, reducing job flakiness during integration testing.

Changes:

  • Set GCP_PREEMPTIBLE: false for the GCP deploy-director task used by IPv4 stemcell tests.
  • Set GCP_PREEMPTIBLE: false for the GCP deploy-director task used by BATS stemcell tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread ci/pipelines/builder.yml
These are not stable enough for testing purposes.
@aramprice aramprice merged commit 0238ab5 into ubuntu-jammy May 7, 2026
12 checks passed
@aramprice aramprice deleted the stop-using-spot-instances branch May 7, 2026 16:56
@github-project-automation github-project-automation Bot moved this from Pending Merge | Prioritized to Done in Foundational Infrastructure Working Group May 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

3 participants